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Two piloted simulations were conducted at NASA’s Johnson Space Center using the 
Cooper-Harper scale to study the handling qualities of the Orion Command Module capsule 
during atmospheric entry flight. The simulations were conducted using high fidelity 6-DOF 
simulators for Lunar Return Skip Entry and International Space Station Return Direct 
Entry flight using bank angle steering commands generated by either the Primary 
(PredGuid) or Backup (PLM) guidance algorithms. For both evaluations, manual control of 
bank angle began after descending through Entry Interface into the atmosphere until 
drogue chutes deployment. Pilots were able to use defined bank management and reversal 
criteria to accurately track the bank angle commands, and stay within flight performance 
metrics of landing accuracy, g-loads, and propellant consumption, suggesting that the 
pilotability of Orion under manual control is both achievable and provides adequate 
trajectory performance with acceptable levels of pilot effort. Another significant result of 
these analyses is the applicability of flying a complex entry task under high speed entry flight 
conditions relevant to the next generation Multi Purpose Crew Vehicle return from Mars 
and Near Earth Objects. 


1 Aerospace Engineer, Advanced Mission Design Branch, Mail Stop EG5; michael.a.tigges@nasa.gov 

2 Aerospace Engineer, Aeroscience & Flight Dynamics Section, Mail Code JE-B225; brian.bihari@escg.jacobs.com 

3 Senior Human Factors Design Engineer, 2400 NASA Pkwy, Mail Stop B2A ; j ohn-paul. Stephens @ lmco.com 

4 Senior Human Factors Engineer, 1290 Hercules Drive, gordon. vos @ wyle. com , Senior Member AIAA 

5 Aerospace Engineer, Flight Trajectory Dynamics & Controls Branch, Mail Stop 210-10; karl. hilimoria @ nasa. gov , 
Associate Fellow AIAA 

6 Aerospace Engineer, Flight Trajectory Dynamics & Controls Branch, Mail Stop 210-10; eric.mueller@nasa.gov , 
Senior Member AIAA 

7 Aerospace Engineer, Integrated GN&C Analysis Branch, Mail Stop EG4; howard.g.law@nasa.gov 

8 Aerospace Engineer, Advanced Mission Design Branch, Mail Stop EG5; wyatt.johnson@nasa.gov 

9 Aerospace Technologist, Crew Systems and Aviation Operations Branch, Mail Stop 152; 
randall.e.bailey@nasa.gov . Senior Member AIAA 

10 Senior Research Engineer, Dynamic Systems and Control Branch, Mail Stop 308; bruce.j ackson @ nasa. gov . 
Associate Fellow AIAA 


1 

American Institute of Aeronautics and Astronautics 



I. Introduction 

H andling qualities are those characteristics of a flight vehicle that govern the ease and precision with which a 
pilot is able to perform a flying task. 1,2 They arise through the interaction of many factors: the stability and 
control characteristics of the vehicle, the automatic control systems that modify these characteristics, the 
interfaces that present flight information to the pilot along with the control mechanisms they use to control the 
vehicle, and all other cues — aural, visual and proprioceptive — that provide feedback to the pilot on his performance 
during the task. Good handling qualities can reduce the amount of training time required for pilots and improve the 
probability of mission success. Continued research allows a deeper understanding of pilot workload and more cost 
effective design for manual operations. While handling qualities standards for aircraft have been used by designers 
for years to ensure piloting tasks are not excessively difficult to accomplish, corresponding standards do not exist for 
spacecraft. Due in part to both the lack of standards as well as the inherent experimental nature of spacecraft, 
handling qualities evaluations must be done for each new spacecraft design, and the earlier in the design process this 
is done the easier it is to remedy handling qualities deficiencies. Studies of spacecraft handling qualities date back 
to the Apollo era, mostly dealing with orbital rendezvous/docking and lunar landing. The space shuttle program 
conducted some handling qualities studies of final approach and landing/rollout. 2 3 In 2007, NASA began another 
effort to study spacecraft handling qualities in preparation for a new generation of piloted spacecraft intended to 



Figure 1. Orion handling qualities evaluations testing program. 

replace the Space Shuttle and travel beyond Earth orbit. 4-11 Although vehicle control during entry is usually done 
with an automated system, Apollo and current-day requirements (e.g. NASA’s Human Rating Requirements, NPR 
8705.2b) stipulate that pilots have the ability to manually control the vehicle’s flight path and attitude in off-nominal 
situations. 12 Atmospheric entry studies from the Apollo era concentrated on evaluating the pilot’s ability to control 
the vehicle to various acceleration levels, determining the cockpit display requirements for controlling to a 
downrange and crossrange target, and development of nominal and emergency procedures. 13-15 Recent NASA 
studies have looked at the ability of a pilot to accomplish two entry tasks: following explicit bank angle commands 
from a backup guidance system; and controlling the vehicle to a precision landing using a schedule of altitude, Mach 
number and energy as function of range. 11 While these studies showed manual control during reentry was feasible 
and determined several types of displays and instrumentation that would provide relatively good handling qualities, 
they did not establish requirements or flight rules associated with the types of skip trajectories required for lunar 
return (a much higher speed return than that seen from low earth orbit), or take advantage of opportunities for more 
precise touchdowns afforded by modern sensors and computers. 

The objective of the handling qualities assessments reported in this paper was to evaluate the atmospheric entry 
handling qualities (HQ) of NASA’s new Orion spacecraft, and specifically the HQ of the Orion Command Module 
(CM) during atmospheric entry returning from a lunar mission or low Earth orbit. The primary goal of the 
evaluations was to evaluate the Guidance, Navigation, and Control (GN&C) design and vehicle configuration 
against the performance requirements as set forth in the Human-System Integration Requirements (HSIR) document 
(CxP 70024 Rev D), and to influence displays and control designs leading into final HQ Verification and Validation 
(see Figure 1). Two evaluations were completed separated by two years, providing opportunity to assess and iterate 
factors affecting manual HQ (displays, pilot inputs, piloting techniques) of a capsule configuration under 
development (since actual hardware and cockpit display configurations will not be ready until at least two years 
following the second study and verification of final HQ is complete). This paper documents a dynamic 
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developmental approach of prototyping concepts utilizing high fidelity simulators in concert with supporting open- 
loop and closed loop offline testing and Monte-Carlo evaluations. 

This study’s original contributions lie in the determination of requirements for flying Lunar Return Skip Entry 
trajectories and the investigation of two new entry guidance schemes: one “backup” system that requires only 
knowledge of attitude, acceleration and initial entry conditions, and a sophisticated numerical predictor-corrector 
algorithm that achieves touchdown accuracies within 5 nmi even in the expected worst case atmospheric and vehicle 
properties dispersions. The next section will describe in detail the dynamics of atmospheric reentry for a capsule 
spacecraft, the guidance algorithms that deliver the spacecraft to the desired touchdown point, and the piloting task. 
Section 3 describes the suite of simulators used during this evaluation and Section 4 outlines the experiment design 
and the configurations tested. Finally, the key results and conclusions are presented in the final sections. 


II. Capsule Entry Dynamics 

The Orion Spacecraft uses a capsule design similar to that used for entry flight during the Apollo Program. 
Although scaled slightly larger to accommodate a larger number of crew, the fundamental characteristics for vehicle 
control during aerodynamic flight are maintained. Like Apollo, the Orion capsule uses the aerodynamic forces 
generated by the flow of air around the blunt body shield to stabilize the vehicle at a desired attitude (angle-of- 
attack) and generate the drag and lift accelerations required for slowing the vehicle down and steering. The angle- 
of-attack is set by carefully packaging the vehicle to attain a desired Center-of-Mass (CM) offset location that 
achieves the desired vehicle pitch trim. Drag acceleration is used to slow the vehicle down, while the lift 
acceleration is used to control the vehicle’s descent rate into the atmosphere. The direction of the lift acceleration 
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Figure 1. Orion capsule dynamics and control. 

relative to the vehicle’s flight path is controlled by firing Reaction Control System (RCS) jets that rotate the vehicle 
about the air-relative velocity vector and achieve the desired bank angle (see Figure 2). During bank maneuvers, the 
vehicle pitch attitude (angle-of-attack) is stabilized by aerodynamic forces and moments which maintain the blunt 
side of the capsule, protected with an ablative Thermal Protection System (TPS), facing towards the plasma of high 
speed energized particles impacting and streaming by the vehicle. The ablative cooling provided by the TPS 
protects the vehicle structure and internal systems and maintains the crew at a comfortable temperature level within 
the cabin. Simultaneously with auto or manual bank attitude control, excessive rates in the pitch/yaw axes are 
damped by RCS firings commanded by the auto flight control system. 

The trajectory control authority available for a capsule vehicle during aerodynamic flight is specified by the ratio 
of lift to drag acceleration, or L/D. Nominal L/D for the Orion capsule is determined by the amount of CM offset, 
and was set within the range of 0.27 to 0.25 for the Handling Qualities Evaluations. This implies that the nominal 
lift acceleration available for the auto/human control system to affect vehicle trajectory was 25-27% of the total drag 
acceleration. Figure 2 shows a resultant combined lift and drag acceleration (or g-load) in the direction of “eye -balls 
in” at the pilot station. Note also that the blunt shield TPS design with optimal mass packaging of the capsule to 
maximize CG offset and thereby L/D, results in an upside-down backwards crew orientation relative to the direction 
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of travel (i.e., the evaluation pilot faces away from the direction of travel with lift pointing down through his feet). 
These factors become important when determining how effectively an evaluation pilot projects simulation results 
into the real world and establishes situational awareness as the capsule is manually flown to the target. 

As the auto/human control system banks the vehicle so that the lift acceleration is directed away from the Earth, 



Figure 3. Entry downrange and crossrange. 

or lift-up (0 degrees bank), the rate of vehicle descent into the atmosphere is slowed, reducing the rate that drag 
increases, and extending the vehicle range. As the auto/human control system banks the vehicle so that lift 
acceleration is directed towards the Earth, or lift-down (180 degrees), the rate of vehicle descent into the atmosphere 
is increased, increasing the rate that drag increases, and decreasing the vehicle range. To simulate a vertically 
neutral or zero lift acceleration mode of flight, the auto/human control system banks the vehicle to the side of the 
direction of flight (90 degrees bank). However, for a lifting body capsule design, any bank away from pure lift-up 
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Figure 4. Lunar Return Skip Entry and ISS Return Direct Entry flight geometry. 


or lift-down results in a lateral acceleration that will push the vehicle off the current groundtrack and permit 
achieving a crossrange landing target (Figure 3). Any crossrange error can be reduced by the auto/human control 
system periodically reversing the direction of the bank angle in a maneuver called a Bank Reversal. The target 
landing accuracy of an Orion capsule using automatic bank angle control of downrange and crossrange, when 
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subject to the full range of atmospheric, aerodynamic, navigation, control, and mass properties dispersions, is within 
10 km (5.4 nmi). 

Two flight scenarios were used to evaluate Orion’s entry handling qualities. The first scenario assessed in the 
First evaluation modeled a skip entry flight for a crew returning from the moon on an ascending approach to the San 
Clemente water landing site location near San Diego, California (see Figure 4). A skip entry flight is broken into a 
high speed first entry phase and a lower speed second entry phase similar to low earth orbit returns. As shown, for 
lunar returns the entry location is driven by the periapse (or antipode) of the Earth Moon orbital geometry at the time 
of lunar departure. The second scenario assessed in the second evaluation modeled a direct entry for a crew 
returning from the International Space Station (ISS) on a descending approach to the San Clemente water landing 
site location. For ISS returns, the entry location is driven by the deorbit maneuver after departing from the Space 
Station. 

As shown in the Figure 4 altitude vs. Downrange Distance insert plots, the difference between a direct entry for 
ISS return and a skip entry flight for lunar return was the amount of lofting that occurs during the entry. The Apollo 
program used a direct entry approach for returning crew from the moon. A skip entry flight has never been flown in 
a manned space flight program therefore manual HQ investigations focused on this flight type in the first evaluation. 
For direct entry flight, the peak accelerations never reduced below 0.2 Gs after Entry Interface (400,000 ft altitude). 
For skip entry flight, the entry accelerations are decreased below 0.2 Gs during the loft phase after first entry 
followed by a second entry and g-onset to final atmospheric descent and landing. The higher energy of lunar return 
enables a lunar return crew to fly extended downrange distances and land in proximity to the Continental United 
States (CONUS) throughout the lunar month. Without the skip entry option, a lunar return crew would be forced to 
land in the Southern Hemisphere for an extended period during each month. 

Entry Guidance Algorithms 

Two guidance algorithms were assessed in these evaluations. The Predictive Guidance (PredGuid) computes the 
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Figure 5. Lunar Return Skip Entry and ISS Return Direct Entry footprint comparisons. 


bank commands for Lunar Return Skip Entry flight and for ISS Return Direct Entry flight. PredGuid is the primary 
guidance algorithm for Orion Entry. A back-up guidance algorithm, Precision Loads Managed (PLM), was also 
assessed. PLM computes the bank commands for contingency flight operations for ISS return. The PredGuid 
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algorithm provides the best footprint performance for entry flight, but requires a full functional navigation system 
(position, velocity, and attitude). PredGuid delivers the capsule precisely to the desired target, but requires the most 
resources (propellant and navigation) and requires about four bank reversals. The PLM algorithm was designed for 
simplicity and robustness and is implemented with only 80 lines of code. PLM guidance requires only bank angle 
attitude and acceleration information and some lookup tables, and executes only a single bank reversal triggered on 
velocity. Ballistic entry (capsule spin stabilized) is a third method of entry for Orion that was not assessed during 
these evaluations. The PLM algorithm provides significantly better performance than a ballistic entry with lower 
acceleration levels seen by the crew and vehicle. Figure 5 shows the landing footprints for these three different 
types of entry guidance options. 

PredGuid software architecture is comprised of three main pieces. The first two distinct pieces are non-flight 
tested software active during the first skip entry phase of flight for lunar returns (PredGuid-Lunar) or the initial 
phase of re-entry from Low Earth Orbit (PredGuid-LEO). During this high-speed early phase of flight, the 
PredGuid-LEO and PredGuid-Lunar algorithms utilize a Numerical Predictor-Corrector (NPC) to determine the 
bank command required to interface into the final main piece of the PredGuid-LEO or PredGuid-Lunar algorithms, 
the vintage Apollo Guidance. The numerical phases are model based and use estimators to determine the vehicle 
aerodynamics and atmosphere; which are used to numerically integrate the vehicle state and compute a bank 
command that achieves range and energy condition (an “energy bucket” or velocity/flight-path angle, v-y manifold) 
targets at Apollo Phase Initiation. Combined with a range estimate provided by the Apollo Algorithm, the PredGuid 
algorithm bank command is embedded in a closed-loop feedback that insures the required range (total range to go) is 
flown to reach the drogue deploy target (see Figure 6). The numerical bank command is recursively computed 
during flight to incorporate and correct for changes in vehicle state, aerodynamics, and atmospheric density. This 
“longitudinal channel” NPC iterative loop insures that the correct amount of energy is depleted from the vehicle 
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Figure 6. PredGuid target range estimation. 
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before initiating the Apollo phase of flight. A lateral channel of PredGuid guidance, which is decoupled from the 
longitudinal channel, performs calculations required for bank reversals (sign changes on the bank command) that 
manage heading error throughout all phases of non-contingency entry flight. 

For contingency entry flight, the back-up Precision Loads Managed (PLM) guidance algorithm could take over 
computation of the bank command. As stated previously, PLM is a simpler analytically based algorithm designed 
for use with minimal information from the navigation system (local attitude and acceleration information). Using a 
set of look-up tables functionalized to acceleration and velocity estimates, a bank command is computed based on 
observed g-load and nominal load comparisons. Only a single bank reversal is commanded on velocity. This 
simplicity is reflected in the bank command profiles shown in the Experimental Design Section IV. 


III. Simulation and Testing Facilities 


Simulation Software 

The core simulation, computer hardware, and software behind these studies was the Advanced NASA 
Technology Architecture for Exploration Studies (ANT ARES) simulation. 16,17 It is a six Degree of Freedom (DOF) 
simulation made up of a collection of libraries and packages. It has been designed to support a wide variety of 
requirements assessments (real-time, non real-time, deterministic, Monte Carlo, etc.), design trades, and operator-in- 
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the-loop evaluations for the Crew Module (CM). ANT ARES is based on Common Model Library (CML) 
architecture and the Trick simulation tool kit which leverages the significant simulation development conducted by 
several organizations across JSC, including the Aeroscience and Flight Mechanics Division (A&FMD), the 
Automation, Robotics and Simulation Division (ER) and the Mission Operations Directorate (MOD). In 2008, it 
won the JSC Exceptional Software Award, the NASA Exceptional Technology Achievement Medal, and the NASA 
Agency Software of the Year (JSC). It is used for Orion GN&C algorithm development and analysis. Although 
primarily a batch simulation, ANT ARES was tailored to be used for real-time pilot-in-the-loop evaluations with a 
Rotational Hand Controller (RHC), notional displays for piloting cues and flight control system configuration, and 
an out the window visual. 

Cockpit 

The ANT ARES simulation was integrated into two separate Reconfigurable Operation Cockpit (ROC) mock- 
ups. The Lunar Return Skip Entry scenarios were assessed in the Mini-Dome (see Figure 7) where evaluation pilots 
sat upright. The ISS Return Direct Entry scenarios were assessed in the Beta-Dome (see Figures 8) which included 
a medium-fidelity seat which allowed evaluation pilots to sit recumbent and have the correct direction of sensed 
acceleration vector for entry. Both ROC mock-ups provided a simulated Orion cockpit environment including 
display units, hand controllers, and window portals. The three display units in each mock-up showed Orion displays 
and overlays. The simulation domes provided image projection that could be viewed through the mock-up windows 
creating an immersive visual environment adding to situational awareness. While the Beta-Dome is better suited for 
entry HQ assessments, this facility was not available during the Lunar Return Skip Entry evaluation. 



Figure 7. Lunar Return Skip Entry in ROC Mini-Dome. 
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Testing Facility -A GPL 

The Advanced GN&C Development Laboratory (AGDL) was a critical facility in the planning stages of the 
evaluation. The AGDL is home to a desktop version of the ANT ARES simulator with cockpit interfaces used for 
initial evaluation development, preliminary testing to solicit astronaut office feedback during scenario development, 
and open loop response examinations for the ISS Return Direct Entry study. The AGDL additionally supported 
prototyping of manual control algorithms used in the selected scenarios, determining the Cooper-Harper criteria, 
developing specific cases to be used for the evaluations, and refining case sets of mass properties, initial vehicle 
state, aerodynamics, atmospheric conditions, display and navigation sensor models, nominal and dispersed initial 
condition sets, and data output parameters for the actual study data runs (refer to appendix for more information). 

Crew Displays 

Two different generations of displays (Figure 9) were used for the assessments, with one display set used for 
Lunar Return Skip Entry tasks, and a separate display set used for ISS Return Direct Entry tasks. The changes 
between these two sets of displays were simply due to display design concepts having matured over time, though 
both display sets conveyed the same basic information. This information included current vehicle attitude, a fly-to 
bank error needle and body-axes rotation rates were presented on an Attitude Direction Indicator (ADI), Mach or 
relative velocity, altitude and altitude rate tapes and digitals and an analog acceleration (g-load) meter. These 
displays gave the crew situational awareness and allowed them to follow the guidance commands and monitor 
vehicle response to guidance and manual commands. 
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Figure 9. Prototype cockpit displays: left (Lunar Return Skip Entry), right (ISS Return Direct Entry). 


Lunar Return Skip Entry Runs 

The ADI ball included a digital readout of the bank errors for values above +/-10 deg and absolute bank 
indicators (yellow diamond indicated guidance command, while blue triangle displayed actual bank from 
navigation). A digital readout of the percentage of Reaction Control System Propellant Remaining and the Range to 
the Target were both displayed. This display was derived from Space Shuttle heritage displays and thus is not 
proprietary. 
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ISS Return Direct Entry Runs 

The display also included a heading error or delta azimuth indicator (AAz) which was specifically added for the 
2010 evaluations based on commentary from the 2008 assessment. This indicator provides the crew with 
information on the approach of the heading error to the corridor boundary which determines the execution of the 
bank reversal command (Figure 10). Additionally, the bank error needle on the ADI ball used a smoother and 
shortest time logic update and had a range of +/- 15deg. This display did not include a digital indicator to show 
errors above 15 deg. However, a second indicator for actual bank error was displayed to the evaluation pilots 



Figure 10. PredGuid lateral heading error corridor used to determine bank reversals. 

utilizing the airplane symbol in the center of the ADI ball. RCS propellant usage and range to target were removed 
for this set of runs since the evaluation pilots could not directly control those parameters, but they will be displayed 
on other crew displays in future studies. Lateral Chanel information detailing time to bank reversals and delta 
azimuth (AAz) were schedule to be on the display but the simulation engineers ran out of time to implement these 
items. The Display Not Active region was not utilized due to time constraints, but was to include: a landing footprint 
plot consisting of constant roll lines, a ballistic impact point, a vehicle capability footprint, a landing site marker, the 
coast line, digital readout of maximum g-load and digital readout of footprint dimensions. The look and feel of this 
display was updated from the Lunar Return Skip Entry display to reflect the current state of the art of the Orion 
vehicle and is thus currently not available to show in this paper. 

Input Devices 

Evaluation pilots used a Rotational Hand Controller (RHC) to send bank-angle rate commands to the control 
system. The ROC facilities contain RHCs manufactured by Measurement Instruments, Inc. (MSI). This 
configuration was used for the Lunar Return Skip Entry evaluations. For the ISS Return Direct Entry evaluations, 
the RHC in the Beta-Dome was modified to have a full scale deflection closer to the final Orion design. In the Beta- 
Dome, the RHC was mounted on an adjustable arm which was attached to the medium- fidelity Orion seat structure. 
The position of the RHC was adjusted to the anthropometry and comfort of each evaluation pilot prior to the start of 
the evaluation. The evaluation pilot’s arm was thus supported behind the upper arm or elbow, along the vehicle x- 
axis, as required for high acceleration phases. The RHC mounting mechanism was not flight-like, but allowed the 
RHC to be located in a representative flight location. The RHC had a ±10% deadband, and commanded a bank- 
angle rate of 20 deg/sec at full deflection. It used a parabolic shaper that was tuned via pre-simulation testing and 
gain sets. Figure 1 1 shows the bank-angle rate commands as a function of stick deflection. 
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Acceleration Forces 

One of the biggest assumptions for the current evaluations was how to handle acceleration forces. For the skip 
entry scenarios, evaluation pilots sat upright in an adjustable chair in the Mini-Dome. The chair height was adjusted 
to place the evaluation pilots at the correct vehicle eye point. For the Direct Entry scenarios in the Beta-Dome, 
evaluation pilots were seated reclined with the gravity vector normal to the chest (x-axis). This added a little realism 
for portions of the evaluations with a lg force in the x-axis, but deviations from lg could not be modeled. As with 
the skip entry scenarios, evaluation pilots were seated at their correct vehicle eye point. Some extreme off-nominal 
evaluation runs exceeded 8g’s as predicted by the ANT ARES simulation. Evaluation pilots were asked to comment 
on the increased complexity of the tasks that would be performed at high g-load levels, but were asked not to 
speculate how these accelerations would impact the handling quality or their workload ratings. 

Auditory Cues 

The evaluation pilots were not provided audio feedback of the Orion CM RCS attitude jet firings during any of 
the simulations evaluated. This issue was identified by evaluation pilots during the Lunar Return Skip Entry 
evaluations as a possible area of improvement. The primary nose RCS jets on the space shuttle can be heard when 
fired; the tail RCS jets and vernier jets are not audible. Since Orion’s jets more closely resemble the space shuttle’s 
vernier jets than the shuttle’s other jets, it is uncertain if Orion’s crew will be able to hear the jets fire. Due to the 
uncertainty, the test team and evaluation stakeholders agreed that no audible cues would be provided to the 
evaluation pilots of jets firing. Unlike the previous round of evaluations, no evaluation pilots commented on the 
lack of auditory cues for the ISS Return Direct Entry evaluation. 

Suits and Gloves 

Space suits, gloves, and helmets were not employed during these evaluations. Evaluation pilots wore typical 
street clothing. Nominally, Orion flight crew would be fully suited for all scenarios evaluated, but suits were not 
incorporated into these evaluations primarily due to suit availability and resources. 

IV. Experimental Design 

The Lunar Return Skip Entry and ISS Return Direct Entry evaluations were performed during the summers of 
2008 and 2010 respectively. The planning and execution of these two evaluations was a collaborative endeavor 
between NASA and the Prime Contractor developing the Orion Crew Exploration Vehicle (Lockheed Martin, LM), 
and included multiple engineering disciplines from each organization. Key roles in the evaluation were performed 
by the Astronaut Office, GN&C Engineers, Human Factors Engineers, Simulation Engineers, and members of the 
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Ames Research Center (ARC) and Langley Research Center (LaRC) Spacecraft Handling Qualities (SHaQ) group 
(see the acknowledgement section for additional contributors). 

Each evaluation was broken into three distinct phases: evaluation development, dry runs, and formal evaluation. 
The evaluation development phase included discussion and assessment of what types of manual control scenarios to 
investigate the identification of appropriate performance metrics for each scenario, and design/development of 
simulations for candidate scenarios. Once the scenarios were simulated and the metrics derived, dry run 
assessments were used to evaluate the scenario simulations and evaluation procedures. Feedback from the dry run 
evaluators was used to improve evaluation pilot briefing packages, scenario-specific piloting metrics, and evaluation 
time estimates. 

Assumptions and Constraints 

The computer simulations, derived from ANT ARES version 9.2.0, were high fidelity and accurate with regards 
to the physics of Orion under various flight conditions and use of actual Orion GN&C flight control algorithms, with 
a few noted exceptions including the omission of propellant depletion. The evaluations included many modeled 
sources for error, including: atmospheric dispersions, aerodynamic dispersions, vehicle mass and center-of-mass 
uncertainty, navigation sensors dispersion (platform alignment, acceleration drift, bias, etc.), navigation state 
dispersion (position and velocity for entry), navigation attitude dispersion, crew module RCS thrust dispersion, and 
control system lag. 

Other than these sources for error, the simulation modeled the Orion flight vehicle as “healthy,” i.e., it did not 
have any system failures or degradation. Thus, from a mission operations standpoint, the performance of manual 
crew handling tasks was assumed to be elective in nature, based on decision criteria other than system 
failure/degradation. 

Entry Dispersions and Cases 

To select the cases to be flown for both skip entry and direct entry, a Monte-Carlo analysis was performed. For 
ISS Return Direct Entry, a 3,000 case Monte-Carlo analysis was performed for both PLM and PredGuid. Data was 
collected on the downrange and crossrange miss at drogue deploy, the propellant consumption, the peak g-load, and 
the Max Fractional g-load (the maximum of the ratio of the g-load durations during flight divided by the maximum 
durations allowable by NASA requirements). A zero value of Max Fractional g-load implies 100% margin from the 
constraint and a value of 1 implies zero margin. The 3,000 Monte-Carlo cases were then reduced to 138 by isolating 
rolling moment coefficient dispersion cases close to +2-sigma. These rolling moment cases were isolated to insure 
that the most significant aerodynamic dispersion affecting vehicle handling qualities was present in the final set, 
since rolling moments directly affect the bank-angle rate seen by the test evaluation pilots. From these reduced 
Monte-Carlo sets, three runs were selected for each guidance with a 1-2 sigma combined error source in Max 
Fractional g-load and propellant consumption. These flights were called “highly dispersed” flights, although none 
were outside of a 2-sigma envelope of dispersions from the mean. Due to the simple nature of PLM, only one 
“worst case” ±2-sigma rolling moment dispersion case (referred to as PLM High Dispersion #2) was chosen by the 
test team to be the only PLM case flown due to the substantially higher than the mean g -loads. The Test Team 
wanted to elicit crew responses for this type of manually guided simulated environment. For the PredGuid 
evaluations, two dispersed flight cases (referred to as PredGuid High Dispersion #1 and #2) and the nominal flight 
case were chosen by the test team. The Lunar Return Skip Entry evaluation used a similar but simplified approach 
for selecting the dispersion cases by using only maximum range miss and maximum fuel usage statistics from a 
3,000 case Monte-Carlo 

Experiment Matrix 

The principal objective of this experiment was to evaluate capsule handling qualities during atmospheric entry 
under manual control with different guidance options and entry conditions. There were two types of entry 
trajectories: Lunar Return Skip Entry and ISS Return Direct Entry. Using predictive guidance, the Lunar Return 
Skip Entry trajectory was flown in two segments: the initial entry into the atmosphere and the second entry 
following the exo-atmospheric skip. For Skip-Entry I, pilots evaluated three scenarios: a nominal scenario and two 
types of dispersed scenarios (worst miss and worst propellant). However, they assigned a single Entry I rating 
(Cooper-Harper, TLX) consolidated across these scenarios. For Skip-Entry II, pilots evaluated only the nominal 
scenario. For ISS Return Direct Entry configurations using predictive guidance, a nominal and two highly dispersed 
scenarios were evaluated. The PLM case was flown using a highly dispersed condition. The various configurations 
of the experiment matrix are depicted in Table 1. 
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Table 1. Experiment matrix 


Return Type 

Predictive Guidance 

Precision Loads 
Managed Guidance 

Lunar Return 
Skip Entry 

Entry I (pre-skip) 
Nominal, worst miss, 
worst propellant 



Entry I (post-skip) 
Nominal only 


ISS Return 

Nominal 


Direct Entry 

High dispersion #1 

High dispersion #2 


High dispersion #2 



Evaluation Pilots 

Astronauts from the Crew Office at the Johnson Space Center served in three different roles during the 
development and execution of these evaluations. Some astronauts were involved in the planning and development, 
some participated in the dry run activities and others were involved as evaluation pilots (i.e. test participants). Only 
astronauts who had not participated in the development or dry run activities served as evaluation pilots. For each of 
the evaluations, two shuttle commanders with test piloting experience served as dry run evaluators. Ten astronauts 
evaluated the Lunar Return Skip Entry scenario, while five assessed the ISS Return Direct Entry scenario. All 
evaluation pilots were experienced astronauts (commander, pilot, or mission specialist) with the majority of them 
(11 of 15) being experienced military test pilots. All evaluation pilots had experienced atmospheric entry in a 
winged spacecraft (Space Shuttle) or a capsule (Soyuz), but not all evaluation pilots had experience piloting these 
spacecraft during entry. Each evaluation pilot was available for about four hours, and this time constraint was 
incorporated into the experiment design. Evaluation pilots were briefed on the evaluation, informed of their rights 
as participants in the evaluation, and asked to sign informed consent forms. The protocol for these assessments was 
approved by NASA’s Committee for the Protection of Human Subjects (NASA’s internal Institutional Review 
Board (IRB) for research involving human subjects). 

Training Procedures 

Upon arrival, evaluation pilots received a detailed briefing on the experiment background and objectives, flying 
task, control system, displays, test matrix, and data collection procedures including a focused briefing on the 
Cooper-Harper rating scale 1 . Including discussion time with the experimenters, this session lasted approximately 
one hour. This was followed by a training and familiarization session (about one hour) in the simulator cockpit, 
where evaluation pilots practiced the flying task for various representative configurations drawn from the test matrix 
until they felt comfortable that most of the learning curve was behind them. 

Piloting Tasks 

The principal objective of this experiment was to evaluate capsule handling qualities during atmospheric entry 
under manual control with different guidance options and entry conditions. The flight segment under study was 
from the entry interface (El) +190 seconds to drogue parachute deployment point at roughly 40,000 feet. These 
assessments were focused on crew module (CM) manual control using different modes of the primary guidance 
flight algorithm, PredGuid (both evaluations), and the contingency mode guidance algorithm, PLM (second 
evaluation), under a Lunar Return Skip Entry mission scenario (the first evaluation) and a ISS Return Direct Entry 
mission scenario (the second evaluation). 
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The first HQ evaluation was focused on the Lunar Return Skip Entry scenario using the PredGuid-Lunar 
guidance option. The nominal commanded and actual bank profiles for an entry which would return the capsule to 
the designated San Clemente Water Landing Area using the PredGuid-Lunar (Skip-Entry) and PredGuid-LEO 
(Direct Entry) guidance options are shown in Figures 12 and 13. Also shown are the areas of manual flight control 
used during the evaluations. Note that manual control was only enabled after the aerodynamic loads on the vehicle 
stabilized the vehicle in the pitch and yaw axes which minimized any attitude cross-coupling between the bank 



Figure 12. PredGuid-lunar entry bank and bank command vs. time from entry interface. 


(manual) and pitch/yaw (auto) attitude control inputs. Bank maintenance task areas are labeled “M” and bank 
reversal management tasks are labeled “R.” For the first evaluation, a lunar skip entry case was selected for 
assessment because it was identified as the most demanding of all bank control scenarios. In addition, with the 



Figure 13. Nominal PredGuid-LEO bank/bank-command, altitude, and g-load plots vs. time. 


evaluation consisting of multiple tasks within the entry case, handling qualities ratings collected in the evaluation 
could be used to assess other scenarios such as the constant bank angle maintenance case. The pilots controlled to 
the desired bank angle command output by PredGuid-Lunar during the high speed segment of Entry I (illustrated in 
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Figure 12) and in another independent run, the lower speed segment of lunar return Entry II, which is similar (but 
not identical) to a typical ISS return (described below). Each of these trajectory control segments were evaluated 
separately to decouple the HQ ratings (thus poor performance in Entry I would not impact performance in Entry II) . 
Entry I was selected for inclusion in the test because it was the most dynamic portion of the Lunar Return Skip Entry 
profile and most critical for successfully achieving the low speed entry conditions required to achieve the landing 
site target of Entry II. Entry II was included since it was similar to an ISS return scenario. Following each Entry I 
run, the trajectory was accelerated through an auto-system controlled exo-atmo spheric (accelerations < 0.05 gs) and 
endo-atmo spheric flight portion of the entry trajectory until the trajectory was terminated at drogue chute 
deployment. Entry II was not a direct continuation of Entry I because the pilot inputs during Entry I would 
dramatically change the Drogue Chute Deployment and it was desired to get a rating on piloting effectiveness of 
Entry I without the complication of a manually controlled Entry II segment. Entry II runs were flown manually after 
auto accelerating a flight through Entry I to the Entry II starting conditions. For Entry II runs only nominal flight 
conditions were flown in Entry II evaluations. All environmental, state vector and vehicle state conditions systems 
performed at their nominal expected values. Entry II started once the loft and exo-atmo spheric phase was completed 
and the capsule started to re-enter the atmosphere. During both entry scenarios, the evaluation pilots were asked to 
make bank control inputs with the RHC to follow the predictive guidance shown on a display page as part of the 
Attitude Direction Indicator (ADI) data while pitch and yaw were automatically controlled. 

The Lunar Return Skip Entry piloting task was broken into a bank angle maintenance subtask and a bank 
reversal management subtask. Table 2 lists the objective performance criteria used in the evaluation for both the 
bank reversal management and bank maintenance subtasks. The reversal management subtasks began once the 
guidance indicated a bank angle correction of more than 100 degrees, which was considered a bank reversal. The 
evaluation pilots were instructed to achieve a bank angle rate of at least 15 degrees per second during gross 
acquisition and not overshoot the guidance target by more than 15 degrees. For bank angle maintenance, evaluation 
pilots were asked to maintain the bank angle within four degrees of the predictive guidance cue. Below Mach 8 on 
the Entry II profile, the performance criteria were not applied due to both the reduced landing zone altering effect 
related to control inputs during that portion of the profile as well as the significantly increased use of propellant 
required to affect bank angle changes at that stage. For this portion of the Entry II profile, the evaluation pilots were 
instructed to fly in a less constrained manner below Mach 8, not to “chase the needles” excessively due to reduced 
command authority and the associated lack of objective performance criteria for that portion. The Mach 25 to Mach 
8 portion of Entry II was thus the only portion with true performance criteria. In essence the pilot was simply 
"following through" and completing the flight as best they could while conserving fuel for the portion from Mach 8 
to Mach 0.7 (where chutes were deployed). 

Table 2. Performance metrics for Lunar Return Skip Entry and ISS Return Direct Entry 


Lunar Return Skip Entry 

Entry I 

Desired Limit 

Adequate Limit 

Entry II 

Desired 

Limit 

Adequate 

Limit 

Maximum Bank Reversal 
Rate 

>15 deg/sec 

>10 deg/sec 

Maximum 

Bank 

Reversal 

Rate 

>15 deg/sec 

>10 deg/sec 

Steady-state Bank Angle 
Tracking Error 

< 4 deg 

< 6 deg 

Steady- state 
Bank Angle 
Tracking 
Error 

< 4 deg 

< 6 deg 

Over-Shoot / Undershoot 

<15 deg 

<20 deg 

Over-Shoot 

/ 

Undershoot 

< 20 deg 

<25 deg 

ISS Return Direct Entry 

Desired Limit 

Adequate Limit 




Steady-state Bank Angle 
Tracking Error 

±6 deg (2 ticks) > 
135k ft 

±9 deg (3 ticks) < 
135k ft 

±9 deg > 135k ft 
±12 deg < 135k ft 




Over-Shoot / Undershoot 

±9 deg (3 ticks) 

±12 deg (4 ticks) 
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The second HQ evaluation built on the lessons learned from the first evaluation, and focused on the ISS return 
flight missions. Both nominal and contingency guidance algorithms were assessed (PredGuid-LEO and PLM) using 
both nominal and dispersed environment and vehicle conditions. The bank/bank-command, altitude, and g-load 
plots vs time for nominal flight are shown in Figures 13 and 14 for PredGuid-LEO and PLM respectively. As in the 
skip entry evaluation, the pilots controlled to the desired bank command attitude output by PredGuid-LEO or PLM 
(not used for Skip-Entry) during the entry. Velocity conditions are low enough for ISS return flight that the lofted 
skip entry design afforded by PredGuid-Lunar was not required. As in the Lunar Return Skip Entry evaluation, the 
simulation was accelerated through an auto-system controlled upper atmospheric portion of flight until the vehicle 
aerodynamically stabilized at the desired pitch trim attitude. This occurred at approximately 3 minutes after El once 
dynamic pressure reached about 10 psf (about 0.2 G). Both PredGuid and PLM cases asked the evaluation pilot to 
fly a descending approach to the San Clemente landing site starting at El +190 seconds down to drogue deployment 
(an altitude of about 40,000 ft - similar to the Lunar Return Skip Entry scenario). When Orion becomes stable, 
entry manual control is reduced from a 3 degrees of freedom (DOF) task (not evaluated) to a single axis (bank) 
piloting task (excessive rotational rates in the pitch and yaw axes are automatically damped). The entry scenarios 
were flown as single-axis control tasks in the Beta-Dome ROC using the RHC to evaluate closed-loop roll control 
(also termed bank control). This occurred at approximately 3 minutes after Entry Interface when the sensed 
acceleration reached 0.2 Gs. 



PredGuid is the nominal guidance system used by the autopilot to fly the entry phase. When flown manually, the 
RHC is used to track the bank angle command by keeping the bank error needles within tolerance. As with the skip 
entry scenario, the HQ entry task was broken into two subtasks of bank angle maintenance, where the evaluation 
pilot was asked to maintain the bank error needle within particular “mental deadbands” and the second subtask of 
manually executing the commanded bank reversals. 

PredGuid helps predict the next bank reversal by displaying graphically Orion’s current location in the lateral 
corridor on the Primary Flight Display (PFD). As the green indicator bug on the current lateral angle display (AAz) 
approaches a corridor indicator, the evaluation pilots know to anticipate a reversal. 

Flying the PLM scenario requires the evaluation pilot to perform the same two subtasks of bank angle 
maintenance and executing bank reversals. The number of reversals is greatly reduced when flying under the PLM 
guidance (only one versus typically four reversals for PredGuid). The evaluation pilot was directed by the guidance 
needle to execute a reversal at a velocity of ~ 18,700 feet per second. The evaluation pilot also performed a ‘roll to 
lift-up’ during the terminal phase of flight. PLM used the same PFD as the PredGuid scenario, but some of the 
PredGuid features such as the lateral angle error and crossrange boundaries were not available to the evaluation 
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pilots. Other than knowing at what velocity the reversals will occur, the evaluation pilots did not have any 
indication of the approaching reversal until the guidance started the reversal command. The bank to lift-up was also 
a guidance discreet event that happened without pilot notification. 

Entry Piloting Techniques 

During the Lunar Return Skip Entry evaluation, pilots were given suggested flight techniques that mostly 
evolved into those provided later for direct entry evaluation. The Cooper-Harper performance metrics (described 
below) for Lunar Return Skip Entry also heavily influenced the flight techniques of the evaluation pilots. 
Evaluation pilots in the ISS Return Direct Entry evaluations were provided with suggested flight techniques based 
upon those previously utilized by the Lunar Return Skip Entry evaluation pilots. 

The autopilot nominally flies the PredGuid system for entry. For this off-nominal manual control evaluation, 
two of the main mission performance metrics for entry were miss distance (distance from drogue deploy target) and 
propellant usage, both of which are resultant values of the total flight, and not parameters that can provide feedback 
to the pilot as an input during flight. The following list of flight techniques was provided to evaluation pilots for use 
in both PLM and PredGuid scenarios, based upon findings in early testing and scenario development: 

• Use conservative stick deflections to save propellant. 

o Since the RHC is a rate-command stick, any change in stick deflection fires jets and uses 
propellant to achieve that new rate. 

o Once a rate is obtained, do not release it until the desired attitude error is achieved. 

• Use the full deadband during bank maintenance 

o Let the error needle drift within the deadband and do not react to “small” errors. 

o Minor excursions outside the deadband are acceptable. 

• Use small rate commands during bank maintenance (<2 deg/sec). Rates less than ~2 deg/sec will utilize 
only a single string of control jets. 

• Avoid ‘bumping’ the stick in and out of detent. 

• Pursue roll reversals at 9 deg/sec. A Monte -Carlo assessment was performed to determine the maximum 
rate required to achieve “good” entry flight performance. This rate determined for ISS entry flight was 
smaller than the 15 deg/sec rate used in the 2008 Evals, required for the Lunar Return Skip Entry scenarios. 

• Use about a 12 deg ‘lead’ point when feathering off of a reversal. 

• Smoothly achieve the desired bank rate command. 

Entry Performance Metrics 

Usage of the Cooper-Harper handling qualities rating scale 1 involves an assessment of pilot compensation to 
achieve a desired or adequate level of pilot- vehicle performance. In this experiment, the piloting task was to follow 
the bank angle guidance presented on the cockpit display. The guidance tracking performance metrics are presented 
in Table 2 for both Lunar Return Skip Entry and ISS Return Direct Entry. The key operationally relevant 
performance parameters for entry trajectories are g-loads, propellant consumed, and range-to-target at drogue 
deployment. These metrics were computed in data post-processing for comparison with benchmark autopilot 
performance, but were not used in the determination of desired/adequate performance for assigning handling 
qualities ratings. 

During the design phase of the Direct Entry evaluation, it was noticed that the autopilot system did not fly the 
commanded guidance as tightly below 135,000 ft as it did above. From a propellant conservation standpoint, this is 
consistent because the total ranging potential reduces as Orion approaches the drogue deployment. This is 
especially obvious as Orion approaches transonic flight and below when the flight path angle becomes very steep 
and the impact of bank on ranging becomes negligible. For ISS Return Direct Entry, the flight techniques and CH 
performance criteria were designed such that if the flight techniques described above were followed and desired 
performance was achieved, then the flight resultant would provide a miss distance and propellant usage that was 
similar to that of the autopilot. 

Data Collection 

The test team collected both subjective and objective data during the execution of these evaluations. The 
Cooper-Harper Handling Qualities Rating Scale, a subjective rating scale, was used to meet the primary objective of 
this evaluation. However, additional subjective data regarding workload and qualitative commentary as well as a 
wide variety of objective data were collected to satisfy secondary objectives of this study. 
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Handling Qualities Ratings 

The assessment of handling qualities was performed using the 1969 Cooper-Harper rating scale 1 . The Cooper- 
Harper rating scale is the most commonly used handling qualities assessment tool and is required by NASA for 
spacecraft human rating 12 . The Cooper-Harper decision tree (Figure 15) begins with an assessment of the vehicle’s 
“Adequacy for Selected Tasks or Required Operation” in which the evaluation pilot decides if the performance 
achieved in a piloting run was desired, adequate or uncontrollable (these are adjectives associated with performance 
and used as anchors at various locations within the scale). Performance is defined as the precision of control with 
respect to flight vehicle movement that a pilot is able to achieve in performing a task. Adequate performance means 
that the end result obtained was sufficient for successful completion of the required operation or task. Desired 
performance means that the end result obtained exceeded the level of performance needed for successful completion 
of the required operation or task. Since these terms (desired/adequate performance) are involved in the core 
decision logic of the scale, the CH performance metrics associated with their definition are key drivers of the rating 
process. Following the initial determination of operational adequacy, the evaluation pilot subject then proceeded to 
the right of the scale into the defined categories of Level 1, 2 or 3. Next the evaluation pilot reviewed the flight 
vehicle characteristics, the demands on the pilot, and determined the final rating. The Cooper-Harper protocol 
allows for half-ratings within a level (1.5, 2.5, 4.5, 5.5, 7.5, 8.5), but does not allow half-ratings across levels (3.5, 
6.5, 9.5). 

In handling qualities experiments, evaluation pilots are typically asked to make a composite assessment of the 


Handling Qualities Rating Scale 


Adequacy for Selected Task 
or Required Operation 


Aircraft Demands on the Pilot in Selected Pilot 

Characteristics Task or Required Operation* Rating 




Excellent 

Pilot compensation not a factor for desired 1 



Highly desirable 

performance 



Good 

Pilot compensation not a factor for desired 1 




Negligible deficiencies 

performance 




Fair -Some mildly 

Minimal pilot compensation required for 

Yes 


unpleasant deficiencies 

desired performance 



Level 1 


Is it 

satisfactory 

without 

improvement? 


Yes 

l 


Is adequate 
performance 
attainable with 
a tolerable 
, pilot workload 
2 

Yes 

l 


H 

Deficiencies 

► warrant 


improvement 


Minor but annoying 
deficiencies 

Desired performance requires moderate 
pilot compensation 

Moderately objectionable 
deficiencies 

Adequate performance requires 
considerable pilot compensation 

Very objectionable but 
tolerable deficiencies 

Adequate performance requires extensive 1 
pilot compensation 



Deficiencies 
require 
improvement 


Major deficiencies 


Major deficiencies 


Major deficiencies 



Adequate performance not attainable with 
maximum tolerable pilot compensation 


Considerable pilot compensation is 
required for control 


Intense pilot compensation is required to 
retain control 


Level 2 



Level 3 


Is it \No Improvement 
controllable? / r madatory 


T 


Major deficiencies 


Control will be lost during some portion of 
required operation 


Pilot decisions 


* Definition of required operation involves designation of flight phase 
and/or subphases with accompanying conditions. 


Figure 15. Cooper-Harper Rating Scale 1 . 


overall performance across all formal evaluation runs for a test configuration. This assessment takes into account 
not just the quantitative evaluation of the performance at the end point (e.g., drogue deployment) but also a 
qualitative evaluation of the manner in which the vehicle gets to the end point. This overall assessment of desired, 
adequate, or inadequate performance is utilized for traversing the decision tree in the Cooper-Harper chart 1 . Pilots 
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use the Cooper-Harper scale to assign handling qualities ratings from 1 (best) to 10 (worst) based on their 
assessment of task performance and required compensation. It is an ordinal, non-linear scale, such that the 
difference between ratings 1 and 2 is not the same as the difference between ratings 3 and 4, thus the data collected 
using the scale must be assessed using non-parametric methods which lack distributional assumptions of normality 
or linearity. Additionally, the ratings are categorized into levels , such that ratings of 1, 2, and 3 on the Cooper- 
Harper scale correspond to Level 1 handling qualities, which are a general requirement for normal operations of 
flight vehicles. Desired performance is necessary, but not sufficient, for Level 1 ratings. Ratings of 4, 5, and 6 
correspond to Level 2 which may be acceptable for some off-nominal conditions, while ratings of 7, 8, and 9 
correspond to Level 3 which is acceptable only for transition to a safe mode after a major failure or disturbance. 

Workload Ratings 

Additional subjective data was collected by the test conductors, including NASA Task Load Index (TLX) ratings 
for workload (Lunar Return Skip Entry evaluation) and Bedford workload ratings (ISS Return Direct Entry 
evaluation). The reason the workload measures were not consistent between the evaluations is because the NASA 
requirements governing workload changed between the two evaluations. For the earlier evaluation, Lunar Return 
Skip Entry, NASA required TLX, but later, prior to the ISS Return Direct Entry evaluation, the NASA requirements 
for workload changed to the Bedford Workload Scale. 

NASA Task Load Index (TLX) 

The NASA Task Load Index 18 is a weighted average of six workload components: physical demand, mental 
demand, temporal demand, performance, effort, and frustration. The relative weighting of these six components for 
the flying task was determined by a pilot questionnaire at the end of the experiment. Each component is assigned a 
rating between 0 and 100. Unlike the Cooper-Harper or Bedford scales, there are no descriptors associated with 
TLX component ratings; there is only a low/high anchoring on the ends of the scale. A rating of zero corresponds to 
low, and a rating of 100 corresponds to high. The rating for performance is subtracted from 100 prior to computing 
the weighted average. 

Bedford Pilot Workload Rating Scale 

The Bedford Pilot Workload Rating Scale 19 (Figure 16) was utilized to assess the workload associated with the 
ISS Return Direct Entry manual control tasks. Workload is an important measure that can contribute to the handling 
qualities ratings and thus is useful to understand. The Bedford Rating Scale was administered exactly like the 
Cooper-Harper Scale. Evaluation pilots were asked to start at the bottom and walk through each logic gate. Unlike 
CH, Bedford is not broken up into levels and half ratings between decision gates are acceptable (e.g. ratings of 3.5 
or 6.5). 
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Yes 



No 



No 


Was it 
Possible to 
Complete the 
Task? 



Workload Insignificant 

WL 1 

Workload Low 

WL 2 

Enough Spare Capacity for All Desirable 
Additional Tasks 

WL 3 


Insufficient Spare Capacity for Easy 
Attention to Additional Tasks 

WL 4 

Reduced Spare Capacity: Additional 
Tasks Cannot Be Given the Desired 
Amount of Attention 

WL 5 

Little Spare Capacity: Level of Effort 
Allows Little Attention to Additional Tasks 

WL 6 


Very Little Spare Capacity, But 
Maintenance of Effort in the Primary 
Tasks Not in Question 

WL 7 

Very High Workload With Almost No 
Spare Capa city . Difficult in Mai ntai n i n g 
Level of Effort 

WL 8 

Extremely High Workload. No Spare 
Capacity. Serious Doubts as to Ability to 
Maintain Level of Effort 

WL 9 


Tasks Abandoned. Pilot Unable to Apply 
Sufficient Effort 


WHO 


Figure 16. Bedford Pilot Workload Rating Scale 19 . 


Objective Data 

In order to meet secondary objectives, several simulation variables were monitored and collected during the 
evaluations. The ANTARES data relevant to the simulation itself (e.g. vehicle state parameters, flight path 
information, etc.) was electronically recorded. The collection of these variables was programmed into the 
simulation by ANTARES programmers and GN&C specialists. This data collection included metrics defined as 
objective performance criteria used in evaluation of the scenarios using CH by categorizing piloting performance for 
various tasks and subtasks for each run into desired, adequate, or failed performance. Relevant data collected 
includes: 

• Timing information (duration of runs, time stamps for hand controller inputs) 

• Inceptor inputs (type, direction, magnitude, frequency) 

• Vehicle states 

o Vehicle location, orientation and trajectory 
o Vehicle velocity and acceleration 
o Vehicle rotational rates 
o Vehicle control errors 
o Coarse versus fine flight control authority 
o Propellant used/remaining 

Evaluation pilots’ verbal comments were digitally recorded throughout the evaluation and also collected by test 
team note takers. The pre-brief was the only portion of the evaluation where evaluation pilots’ comments were not 
recorded. 
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Data Collection Procedures 

Each pilot encountered the various experiment configurations in a different sequence. In order to avoid bias, the 
evaluation pilots were not told whether the experiment scenario was nominal or dispersed. For each test 
configuration in the experiment matrix, the pilot flew two or three formal evaluation runs. At the end of each run, 
relevant performance data were displayed to the pilot and experimenters. 

After making a composite assessment of the overall performance across the formal evaluation runs for a test 
configuration, pilots assigned a handling qualities rating for that test configuration, as well as a workload rating 
(using NASA Task Load Index 18 for Lunar Return Skip Entry and Bedford Workload Scale 19 for ISS Return Direct 
Entry). As appropriate, evaluation pilots provided qualitative comments about the test configuration they had just 
evaluated. All evaluation pilot commentary was recorded on electronic media while the experimenters separately 
noted key comments and observations. 

After all test configurations had been evaluated, there was a debrief session. Before any scripted questions were 
asked, the evaluation pilot was asked for observations and comments on the scenario. Then the evaluation pilots 
filled out a one-page questionnaire designed to elicit high-level comments on cockpit displays, out-the- window 
displays, guidance cues, control response, and experiment design. This was followed by a discussion with the 
experimenters on the pros and cons of the various test configurations. 

V. Results and Discussion 

For each task flown during this investigation the evaluation pilots provided Cooper-Harper ratings and workload 
ratings specific to each of the experiment configurations, and overall impressions of the manual entry tasks. For 
each run, time histories of numerous simulation variables were recorded along with key performance parameters at 
drogue deployment. These results are presented in this section, categorized by data type: Cooper Harper ratings, 
workload ratings, pilot comments, performance parameters. 

Cooper-Harper Ratings 

Appropriate statistical methods for analyzing and displaying Cooper-Harper ratings are constrained by the 
ordinal nature of the ratings. As there is not a linear or other parametric relationship implied across the rating 
spectrum, only nonparametric ordinal methodologies are acceptable for analysis of Cooper-Harper ratings data. For 
this reason, results of this evaluation have been assessed through the use of fundamental graphical and numerical 
techniques which take these constraints into consideration. Therefore, Figures 17 and 18 present the handling 
qualities results via bubble charts for the various experiment configurations where the size of the bubble for a given 
rating value indicates the number of evaluation pilots who assigned that rating, while the star symbol indicates the 
median rating for that category. 

Lunar Return Skip Entry 

The Lunar Return Skip Entry scenarios evaluated both the initial (Entry I) and second entry (Entry II) portions 
of a high speed Lunar Return Skip Entry profile, with the second phase of entry being very similar to an ISS Return 
Direct Entry profile. Both phases of entry required two types of sub-tasks: a gross acquisition sub-task which 
required the evaluation pilot to acquire a guidance specified bank angle via a large bank angle change (often a bank 
reversal), and quasi-steady state bank angle maintenance sub-task which required the evaluation pilot to closely 
follow the bank angle being commanded. Both sub-task and composite ratings were collected, and they are shown 
in Figure 17. Both Entry I and II resulted in ratings that were a mix of Level 1 (HQR’s from 1-3) and Level 2 
(HQR’s from 4-6). Ratings for the Entry II (post-skip) were somewhat better than for Entry I (pre-skip). The 
evaluation pilots generally met the Cooper-Harper performance metrics and achieved acceptable final landing 
profiles. They indicated that the guidance indicators on the displays were ‘jumpy,’ ‘erratic,’ and difficult to follow 
with comfort or precision. Pilots also indicated that they needed some sort of guidance/bank reversal predictor on 
the display to indicate when significant control inputs (e.g. bank reversals) would be needed. 

Direct Entry 

The Cooper Harper Ratings for PLM were all within the HQR 1-3 rating range indicating consistent Level 1 
handling qualities performance. The PredGuid tasks included a mix of Cooper Harper Level 1 and Level 2 ratings 
(Figure 18). The median Cooper Harper rating for the nominal PredGuid task was a rating of 3, while both 
dispersion cases had median Cooper Harper ratings of 4. The evaluation pilots generally met the Cooper-Harper 
performance metrics and achieved acceptable final landing profiles. These results suggest that the precision loads 
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managed direct entry required minimal pilot compensation, while the predictive guidance tasks required more pilot 
compensation. Indeed, the Cooper Harper ratings for PLM were the best for all of the entry profiles examined, 
though, as discussed later in the comments section, evaluation pilots were concerned that it was also associated with 
the highest g-loading, and that they would have preferred the PredGuid control option due to its lower g -loading and 
increased accuracy. Amongst the predictive guidance cases, nominal conditions still had a median value suggesting 
Level 1 handling qualities were demonstrated, though when highly dispersed conditions were tested they elicited 
Level 2 handling qualities from the vehicle. 



Lunar Return Skip Entry 

The NASA TLX Workload ratings obtained during the Lunar Return Skip Entry testing suggest that for both 
entry profiles (Entry I and Entry II) the primary workload components were: mental demand, effort, frustration, and 
temporal demand. However, median values for even these components were fairly low as evidenced in Figure 19 as 
were the weighted overall TLX scores. Even lower ratings were provided for physical demand and performance 
workload components, suggesting they were secondary workload contributors. The individual component ratings 
and overall TLX rating for Entry I and Entry II were very similar. Subjective comments associated with these 
ratings suggest that the workload, low as it was, was nonetheless driven by a lack of adequate guidance displays. 
The majority of subjective comments indicated that improved displays and predictive guidance would reduce their 
workload scores. 

Direct Entry 

The Precision Loads Managed ISS Return Direct Entry task resulted in highly favorable Bedford workload 
ratings, all in the range of 1-3 on the Bedford scale (Figure 20). However, as with the Cooper Harper Ratings, 
Predictive Guidance ratings for workload were slightly higher than PLM, with ratings for nominal conditions in the 
3-5 range, and highly dispersed conditions resulting in ratings in the 3-6 range. The PLM workload results were 
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satisfactory with the current design; however the predictive guidance workload ratings suggest that modifications 
are needed to improve workload during off-nominal dispersed tasks. 
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Figure 18. Handling qualities ratings for ISS Return Direct Entry evaluations. 


Pilot Comments 


Lunar Return Skip Entry 

Comments for both Entry scenarios I and II were fairly consistent, with a majority of evaluation pilots 
recommending and suggesting improvements to the piloting displays; many of which were incorporated into the ISS 
Return Direct Entry displays. The Cooper-Harter related comments from evaluation pilots were also consistent, 
both across the two entry profiles as well as amongst evaluation pilots. Evaluation pilots indicated that the guidance 
indicators on the displays were ‘jumpy,’ ‘erratic,’ and difficult to follow with comfort or precision. Astronauts also 
indicated that they needed some sort of guidance bank reversal predictor on the display to indicate when significant 
control inputs (e.g. roll reversals) would be needed. Six evaluation pilots commented that they felt that modifying 
the displays to make these two improvements would decrease their pilot compensation and improve or secure their 
Cooper-Harper Ratings as Level 1. Additional comments included feedback on RCS jet activity (due to the lack of 
auditory feedback in the simulation), improvement requested to the RHC response characteristics such as dead-hand 
and gains (indicating that the design used in this assessment made it difficult to ‘feather out’ bank-rate commands), 
and that performing this simulation under g-load would be valuable. Evaluation pilots noted that the vehicle 
responded as desired, but the guidance display was challenging to follow in its notional representation. 

Evaluation pilots also reported that the HQR’s were driven higher (worse) largely due the displays and guidance 
indicators, but not due to the control responses of the vehicle. Orion responded predictably, but the displays and 
guidance were too erratic and jumpy to follow precisely, resulting in more pilot compensation. Evaluation pilots 
indicated that had the displays been higher fidelity and more highly refined that HQR’s in the Level 1 category (1-3) 
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would likely be achievable. The general findings from the entry scenarios were that handling qualities were tightly 
integrated with displays, and that the ratings could be significantly improved through better display development. 
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Direct Entry 

Evaluation pilots commented specifically on either PLM or PredGuid, but they also had many comments that 
applied to both guidance systems. One of the major issues identified with the entry PFD during the skip entry 
evaluation was that the bank error needle was “jumpy.” A smoother was added for the ISS Return Direct Entry 
evaluation to solve this issue. No evaluation pilot identified the needle motion as an issue, and when evaluation 
pilots who had also participated in the skip entry study were asked about the needle, they all agreed that the problem 
was solved. 

Several evaluation pilots commented about needing to evaluate alternative RHC shaping functions. The RHC 
shaping function that was developed for the Lunar Return Skip Entry evaluations was incorporated into the ISS 
Return Direct Entry study. After investigating the issue after the ISS Return Direct Entry evaluation, it was 
discovered that the direct entry evaluation used a stick with less throw than the skip entry evaluation to mimic the 
Orion baseline. This reduced throw compressed the ISS Return Direct Entry deflection curve reducing the 
resolution at the low end of the stick, which resulted in evaluation pilots requesting more resolution in the low end 
of the stick deflection profile to help achieve desired small bank rates. 

Most evaluation pilots desired a warning of the “bank to lift up,” which was present in both PredGuid and PLM. 
The desire was to have an indication of bank to lift up similar to that of the bank reversals (on the AAz indicator). 

In the Lunar Return Skip Entry evaluation, evaluation pilots were provided with both a graphical bank error needle 
and a digital reading of the actual bank command. Because the bank command was not present in the ISS Return 
Direct Entry evaluation, several evaluation pilots requested that adding the actual bank command value to the 
display format should be evaluated. 

Evaluation pilots also were given an indication when a reversal was approaching by either the AAz or velocity, 
but the guidance system was not always predictable in the direction of the approaching reversal. Evaluation pilots 
stated that knowing the direction of the reversal in advance would allow them to better manage their bank 
maintenance and reduce propellant usage by not correcting minor bank errors in the opposite direction of a pending 
reversal. Finally, during the ISS Return Direct Entry evaluation the Time to Roll Reversal digital display was not 
operational. Evaluation pilots commented that functionally incorporating this display would be useful. 
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PredGuid Specific Comments 

Most evaluation pilots commented that PredGuid was not as predictable as PLM and induced an increased 
workload level below 100,000 ft. Several evaluation pilots asked about removing the final reversals since they did 
not seem to impact miss distance, but increased workload and consumed increasingly more propellant. One of the 
most consistent comments was directed to the way PredGuid transitions from inertial velocity (Vi) to relative 
velocity (Vr). At about 12,000 fps relative speed, PredGuid switches from using Vi to using Vr. During this 
transition, the guidance system suddenly determines that the vehicle is traveling slower than expected, so the 
guidance commands a compensating bank to lift vector up. Depending on the magnitude of the bank error present 
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Figure 20. Bedford workload ratings for ISS Return Direct Entry. 


during this transition, the bank angle error needle can peg, causing the evaluation pilot to think a reversal is 
occurring. Depending on the case being flown, the Vi to Vr transition sometimes occurred right before a reversal. 
Most evaluation pilots were unsettled by this change and asked that it be feathered in over time. The AAz would 
rescale as the corridor became smaller towards the end of flight. Several evaluation pilots commented that the 
rescaling caused the evaluation pilots to divert their attention from their current task to view the AAz. Since the 
rescaling seldom occurred around a reversal, the evaluation pilots saw this rescaling as an unnecessary distraction. 

PLM Specific Comments 

All evaluation pilots commented that PLM was simple and predictable. They also commented that the higher g- 
loads experienced during PLM would influence their task performance, though again this may have been an elicited 
concern since the g-loading for this PLM case was the chose because it was the worst case seen during a 3000 
sample Monte-Carlo analysis of g-loading. However, due to the g-loading concerns several evaluation pilots 
recommended that this task be evaluated in a centrifuge. Also, with the simplicity of this guidance system, AAz was 


24 

American Institute of Aeronautics and Astronautics 


not available to indicate when the reversal was approaching. In other comments, several evaluation pilots asked for 
a display warning of the approaching reversal, and some evaluation pilots commented that the reversal could be 
predicted because it occurred at the same velocity regardless of the flight profile. An indication of the approaching 
bank to lift-up was also desired. 

Performance Metrics 

In addition to the ratings and comments thus far presented, the several objective measures of performance 
including range to target (miss distance at drogue chute deploy), propellant usage, and g-load were important in the 
assessment of the Orion vehicle’s manual control capability and handling qualities. Out of these three metrics, only 
current g-load was available via real-time display format to the evaluation pilots. Evaluation pilots were informed 
of miss distance and propellant usage only following their rating and comments at the completion of the study. As 
was mentioned in an earlier section, the Test Team developed several flight techniques to help the evaluation pilots 
maintain similar performance to the autopilot systems. Figure 21 shows the median miss distance for each entry 
case of evaluation pilots compared to the autopilot flying the same case. For miss distance, the evaluation pilot 
median value was less than the autopilot for all cases. Since PredGuid is constantly making corrections to guidance 
based on the evaluation pilot’s bank commands, PredGuid has a small miss distance. Figure 5 shows the results of a 
Monte Carlo simulation comparing the landing locations of the three different guidance systems (PredGuid, PFM, 
and Ballistic). No evaluation pilot exceeded the NASA limit of 5.4 nmi (10 km) miss distance for PredGuid. For 
PFM, the variance between evaluation pilots was much greater than PredGuid. Miss distance ranged from 2.4 nmi 
to 62.5 nmi for PFM for evaluation pilots with the autopilot missing by 17.9 nmi. The PFM data for miss distance 
was not normally distributed across the evaluation pilots, but the majority of data runs (6 of 10) had smaller miss 
distances than the auto-pilot run. It should also be noted that the PFM case chosen for this evaluation was the worst 
3-sigma dispersion case picked from the runs graphed in Figure 4 and is not required to meet the same 5.4 nmi 
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Figure 21. Median miss distances for Lunar Return Skip Entry (left) and ISS Return Direct Entry (right). 

NASA requirement. 

Propellant usage was important to track during the entry scenario to ensure that Orion would have enough 
remaining fuel to perform attitude corrections under the chutes to assure proper orientation for landing. Figure 22 
shows the median evaluation pilot propellant usage for each entry case. The evaluation pilots used more propellant 
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than the autopilot while flying PredGuid. While the majority of evaluation pilots used more propellant than the 
autopilot, for every case, at least one evaluation pilot used less fuel than the autopilot system. Several caveats need 
to be used to interpret this finding. Evaluation pilots were not given feedback on their propellant performance until 
the completion of all their data runs, two of the three PredGuid cases were the worst two cases out of the 3 -sigma 
Monte Carlo simulation, and the maximum propellant usage for the nominal case was about half the recommended 
propellant budget. For PLM, the median propellant usage was slightly better for the evaluation pilots than the 
autopilot. Six out of the 10 data runs out performed the autopilot. It could be rationalized that an evaluation pilot 
could decrease propellant usage at the cost of miss distance, but this was not the case for this study. Three out of the 
four PLM data runs with higher than autopilot propellant usage also had higher autopilot miss distances. In fact, 
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Figure 22. Median propellant usage for Lunar Return Skip Entry (left) and ISS Return Direct Entry (right). 

propellant usage was not highly correlated to miss distance for any guidance system evaluated (all R 2 < 0.48). This 
is a predictable result, since the steady-state bank angle tracking error criteria defined in Tables 2 and 3 were 
developed from Monte-Carlo simulations to insure that miss distance would not be affected. However, if a pilot 
“over controls” within these defined criteria, the propellant consumption can be significantly increased without any 
improvement in range performance. Also, the degraded navigation system assumptions used in the PLM 
formulation does not allow for a directly correlated range improvement with tighter deadbands on bank angle 
control. 
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The entire entry g-load profile was recorded for each data run, but Figure 23 shows the median peak g-load for 
the evaluation pilots against the autopilot. As can be seen, the evaluation pilots’ performance was very similar to the 
g-load during the evaluation, it should be noted that the PLM case flown was the worst 3 -sigma dispersions case out 
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Figure 23. Median peak g-load for Lunar Return Skip Entry (left) and ISS Return Direct Entry (right) 

of the 3000 Monte Carlo simulations run during the evaluation development phase. A nominal PLM contains peak 
g-loads closer to 4.5g, not the 8g seen in this evaluation. Evaluation pilots were asked to monitor and were verbally 
reminded about their current g-load throughout each run. Many evaluation pilots commented that the high g-loads 
seen in the evaluation would impact their performance; impacts that could not be simulated in this evaluation. 
Another area of concern was the PLM bank to lift up, which occurred during the peak-g onset for this case. Some 
evaluation pilots stated that they were not sure how accurately they could hold a specific bank rate during this high 
g-load state. 


VI. Conclusions 

The Lunar Return Skip Entry and ISS Return Direct Entry findings suggest that the pilotability of the Orion 
Crew Exploration Vehicle under manual control is both achievable and provides adequate performance (miss 
distance and propellant usage, g-loads) with acceptable levels of pilot effort. The ISS Return Direct Entry Cooper 
Harper ratings and Bedford workload results both indicated that the Precision Loads Managed (PLM) guidance 
option offered better handling qualities and lower workload requirements than the Predictive Guidance (PredGuid- 
LEO) control option, but it is important to note that evaluation pilots still preferred the PredGuid control due to its 
lower g-loading profile and improved landing accuracy. As the Orion displays and controls continue to mature, 
handling qualities of the Orion vehicle will continue to be refined to farther improve handling quality ratings. Of 
interesting note is that prior to these tests, the Lunar Return Skip Entry was considered by many to be too 
challenging for humans to manually pilot, and though these findings suggest otherwise, it must be remembered that 
these tests were not done under g-loading. Indeed, the PLM guidance option may have obtained the best Cooper- 
Harper ratings, but it also was associated with the highest g-loading of the various scenarios tested, and repeating 
these evaluations under conditions of simulated g-loading could be of significant value and provide a greater fidelity 
of results. Another interesting note of consequence is the recent announcement by NASA and the United States 
Government that the Orion Crew Exploration Vehicle has been chosen to be NASA's new Multi-Purpose Crew 
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Vehicle (MPCV) for use in future exploration of space within the solar system. Orion's designation as the MPCV is 
highly relevant to these findings in that the high speed Lunar Skip Return Entry profile tested here is also 
generalizable to future high speed returns from Near Earth Objects and other planets such as Mars, thus these results 
will have direct impact on future mission planning and return capabilities. 
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Appendix: Significant offline facilities and simulation tools 

As noted in Section III, the Advanced GN&C Development Laboratory (AGDL) facility pictured below was 
used for significant off-line and real-time development work for each of the studies presented in this paper. With 
the simplified window views and seating, and away from the restrictive capsule environment, the engineers and 
pilots could more carefully examine the mathematical models (i.e. the proposed vehicle’s responses). The external 
visual displays also provided test engineers with both pilot and chase plane views. This simple configuraion 
increased the opportunity to understand the capsule’s dynamics and what information the cockpit displays really 
needed to convey to the pilot; and additionally to show the pilot “out the window cues.” The same visual screen’s 
output could be quickly changed to plot vehicle and trajectory data to show the subject pilots the mathematical 



Figure 24. AGDL facility set-up 

response of what he or she was seeing on the cockpit displays and out the window cues. 

The facility in effect, allowed engineers and pilots to down select from off-line Monte Carlo statistical runs 
which the team thought the test subjects would be interested, while allowing the pilots to assist in display and 
technique development for use in the formal ROC facility HQ studies. 

Prior to the formal testing in the ROC, the AGDL lab was utilized for display design, implementation and testing 
for both the Lunar Return Skip Entry and ISS Return Direct Entry evaluations. Test case selection, test case 
procedures and performance parameters were also examined in detail in the AGDL prior to formal testing. Dry runs 
for all scenarios were tested by engineers and pilot representatives. The work also gave the evaluation team a 
chance to begin to think about the problems of projecting from fixed base simulator into the real world environment 
(i.e. flying in a capsule upside down and backwards while experiencing high g-load). 

Before the ISS Return Direct Entry study, the AGDL facility’s contribution to the HQ community as well as the 
ROC studies became more significant. In an effort to develop understanding of the vehicle’s response for future 


28 

American Institute of Aeronautics and Astronautics 



training as well as systems development, a set of open-loop response test cases was performed. A set of test 
conditions from the off-line Monte Carlo studies were selected, in which the pilot would perform a step input of 
specific amplitude and frequency using the RHC. The input responses were inserted in flight critical trajectory areas 
to compare the capsule rotational and translational response to other aerospace vehicles and aircraft. This open loop 
testing covered both the primary (PredGuid) and backup guidance algorithms (PLM). For each test condition the 
pilot would input a specific RHC step input ranging in stick deflection from Va to full stick throw. The four different 
test conditions were selected as listed below: 

1. Start of deceleration (> 0.2g’s) 

2. After the ramp up to one g (>1.0g’s) 

3. Near peak deceleration (5.0 g’s for PredGuid , ~9.0 g’s for PLM) 

4. As dynamic pressure began to fall off and prior to chute deployment were reached ( <1.5g’s) 

Step inputs of one-quarter, half, and full inceptor deflections were flown. The various magnitudes were used to 
test for non-linearity. The resultant roll rate responses to inceptor step inputs were analyzed following standard 
flight test practices in compliance with MIL-HNDBK-1797: 

1 . The step input initiation time was defined as the mid-point of the stick input between the zero input and the 

input size. 

2. From the roll rate response, the maximum slope intercept was determined. The effective time delay was 

computed as the time difference between the maximum slope intercept and the step input initiation. 

3. From the roll rate response, the effective roll time constant was calculated. The effective roll time constant 

was computed as the difference between the time to reach 63% of the steady-state roll rate and the 

maximum slope intercept. 

4. The ‘time-to-roll 30 degrees’ was computed. The 4 time-to-roll 30’ was computed as the difference between 

the time to reach 30 degrees of bank angle and the step input initiation. 

The combination of the open and closed-loop testing indicated several interesting finding for the Orion/MPCV 
and possibly future spacecraft design and development: 


1. Time constant response for a Reaction Control System (RCS) based system will change particularly with the 

level of thruster firing, i.e. if two levels of thruster authority are available a small command input versus 
large command inputs will have different time constants of response. Commands which produced single 
string jet firings had twice the time constant for delay as those producing full thruster level firings. 

2. The applicability of flight phase category and aircraft classifications used in the traditional military handling 

qualities standards (e.g., MIL-HNDBK-1797) is unclear. Comparisons were made assuming that the 
spacecraft was analogous to either a Class II (medium weight) or Class III (heavy weight) fixed-wing 
aircraft with low-to-medium maneuverability. The flight phase category was assumed to be Category B - 
“Those non-terminal Flight Phases that are normally accomplished using gradual maneuvers and without 
precision tracking, although accurate flight-path control may be required.” The general level of rotational 
response of the capsules is similar to such aircraft as the SR-7 1 at high Mach numbers 22 . 

3. The capsules time from rotational response to significant flight path response was much longer than aircraft, 

(.5-3.0 vs. 12-15 seconds) but this may not be significant for HQ... The capsule entry maneuver may in 
effect be like a sluggish sailboat being docked with a very low time constant of trajectory response, which is 
however adequate for a trajectory which takes place very slowly over the space of the vehicles movement.. 
The capsule flight trajectory gradually changes but only has to change gradually over a 30 to 15 minute 
entry period for acceptable pilot response. Thus what would seem to be a totally unacceptable level of 
translation response is in fact acceptable. Note the acceptable results in ROC testing. 

4. The HQ metrics developed in the AGDL lab, fuel used, peak g’s, and position with respect to the landing 

target proved acceptable for a task in which the pilot was in effect flying instruments with his body totally 
disoriented from the direction of flight. Similar metrics are used in studies in which the pilot faces the actual 
vector of flight direction and is visually oriented easily to the local vertical, target frame i.e. landing an 
aircraft. The metrics determined in AGDL were used without further changes in the ROC. 

5. Finally, it was found first in the ROC then confirmed in AGDL lab, that the changes in the RHC originally 

recommended for Orion, were not as acceptable as the original RHC bank command profile used in the 
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Lunar-Return Skip Entry which had been developed from the Space Shuttle. Note AGDL did not reveal the 
same level of objectionable behavior to the changes made between studies but it did confirm that Shuttle 
controller profile was preferred, and when this was retested in ROC both simulations concurred. So the desk 
top tool is not the perfect filter. What however may be most significant is that both simulations showed that 
the shuttle profile originally developed for atmospheric landing flight was preferred for the capsule single 
axis entry instrument control of bank, two very different piloting tasks. This result may just reflect pilot 
experience preference or be the first indications that there are certain ideal or preferred controller profiles for 
rotational rate control of vehicles by human beings when using palm pivot rotational controllers. 

In summary, the finding from the testing confirmed the effectiveness of a simple fixed base table top man in the loop 
interfaces with high fidelity (visual image, not necessarily image projection) and flexible visual displays. If the 
facility allows both pilot and chase plane views, and mathematical plotting of significant dynamic response and 
statistical parameters, it is a very effective tool for initial system engineering and handling qualities development. 
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